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This  paper  investigates  competing  influences  of  display  content  and  clutter  on  pilot  performance  during 
flight  guidance  and  target  acquisition  phases  of  air-to-ground  targeting  missions.  Based  on  interviews  with 
F/A-18  pilots,  a  cognitive  process  model  is  presented  to  help  understand  how  pilots  use  and  transition 
between  internal  and  external  sources  of  information  to  support  decision-making  and  aircraft  control. 
Experiments  were  conducted  in  which  subjects  flew  targeting  missions  using  a  flight  simulator  connected 
to  the  Navy’s  FalconView  moving-map.  In  one  experiment,  subjects  referred  to  three  versions  of  the 
display:  1)  a  detailed  map  overlaid  with  critical  mission  symbology,  2)  the  map  only,  and  3)  mission 
overlays  only.  Flight  guidance  and  target  acquisition  performances  were  best  with  the  overlays-only 
display.  Performance  was  comparable  or  worse  with  the  combination  display  and  significantly  worse  with 
the  map  only,  suggesting  that  the  distraction  of  map  clutter  countered  the  benefits  of  map  content. 


INTRODUCTION 

A  critical  issue  in  military  aviation  is  the  pilot’s  ability  to 
transition  between  internal  (head-down,  instrument-driven) 
and  external  (head-up,  out  of  the  cockpit)  guidance.  Military 
pilots  interviewed  for  this  project  reported  difficulties  during 
this  transition,  particularly  during  time-critical,  air-to-ground 
missions  such  as  Close  Air  Support  (CAS)  and  Forward  Air 
Control  (FAC). 

Air-to-ground  missions  require  precise  flight  guidance  to 
reach  the  target  area  and  timely,  accurate  target  acquisition  to 
search  for,  locate,  and  identify  the  target.  These  mission 
phases  are  interdependent:  errors  during  flight  guidance  may 
result  in  failure  to  acquire  the  target;  conversely,  flight 
guidance  is  a  series  of  target  acquisitions  as  a  pilot  searches 
for  “lead-in”  features  along  the  route  (Conejo  and  Wickens, 
1997).  Flight  guidance  is  supported  by  a  map  or  electronic 
moving-map  display  (internal  guidance  cues),  while  target 
acquisition  relies  on  intelligence  -  often  communicated  by 
troops  on  the  ground  (e.g.,  FAC-G)  or  other  aircraft  (FAC-A) 
-  and  focuses  on  external  cues.  The  pilot  must  correlate 
internal  and  external  guidance  cues  throughout  the  mission  -  a 
process  termed  navigational  checking  (Wickens  and  Long, 
1994)  -  with  the  focus  on  external  cues  commanding  more 
attention  as  the  target  area  approaches. 

The  Navy  has  used  the  cockpit  moving-map  display  as  a 
situational  awareness  tool  for  over  a  decade.  Current  systems 
display  raster  images,  such  as  scanned  charts  and  satellite 
imagery.  Future  systems  will  use  vector  databases  in  which 
individual  map  features  can  be  manipulated,  allowing  the 
display  to  be  customized  and  queried  (Lohrenz,  et  al.,  1997). 
As  new  data  becomes  available,  there  is  a  tendency  to  present 
everything  that  might  be  of  interest.  These  new  displays 
introduce  human  factors  issues  concerning  the  pilot’s  ability  to 
access  and  interpret  the  displayed  information.  Many  studies 
have  linked  display  complexity  to  pilot  performance  (e.g., 


Aretz,  1988).  Display  “clutter”  -  including  unwanted  noise 
and  overcrowding  of  otherwise  important  information  -  is 
thought  to  disrupt  a  pilot's  visual  attention,  resulting  in  greater 
uncertainty  about  target  locations.  A  premise  of  this  paper  is 
that  display  clutter  can  hinder  a  pilot’s  ability  to  correlate 
internal  and  external  guidance  cues. 

Three  F/A-18  pilots  were  interviewed  to  discuss  how  the 
moving-map  display  supports  their  missions  and  how  it  could 
be  improved  (Lohrenz,  2003).  The  pilots  identified  the 
moving-map  as  one  of  several  channels  of  information  they 
use  to  make  decisions  during  time-critical  air-to-ground 
missions.  Figure  1  presents  a  cognitive  process  model 
developed  from  these  interviews  to  help  visualize  how  pilots 
integrate  the  often-disparate  sources  of  information  to  support 
decision-making  and  aircraft  control.  The  model  includes 
1)  the  pilot’s  preconceived  mental  model  of  the  mission;  2)  a 
2D  internal  visual  channel  (moving-map,  flight  instruments); 
3)  a  3D  external  visual  channel  (real-world)  supplemented  by 
a  head-up  display  (HUD);  and  4)  an  aural  channel  for  radio 
communications  (e.g.,  with  a  FAC).  The  proposed  model 
reinforces  the  importance  of  a  pilot’s  ability  to  transition 
between  internal  and  external  guidance.  Clutter  in  any  of 
these  information  channels  is  expected  to  hinder  the  pilot’s 
ability  to  correlate  cues,  impeding  mission  success. 

The  next  sections  present  an  experiment  to  investigate 
competing  influences  of  display  content  and  clutter  on  a 
pilot’s  ability  to  correlate  internal  and  external  cues,  measured 
by  subjects’  performance  and  workload  during  flight  guidance 
and  target  acquisition  phases  of  a  simulated  air-to-ground 
targeting  mission  using  three  moving-map  displays  (figure  2): 

a.  Combination  display:  a  topographic  map  with  graphic 
overlays  of  flight  path  and  target  location, 

b.  Map-only  display:  the  topographic  map  with  no  overlays, 
and 

c.  Overlays-only  display:  the  mission-specific  graphic 
overlays  with  no  underlying  map. 
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Figure  1.  Cognitive  process  model  of  pilot  integrating  various  channels  of  information 
to  support  decision-making  and  aircraft  control. 


Figure  2.  Three  experimental  display  cases:  a)  combination;  b)  map  only;  and  c)  overlays  only. 


METHODS 

Twelve  volunteers  (nine  male,  three  female)  were 
recruited  from  the  Naval  Research  Laboratory  Mapping, 
Charting  and  Geodesy  Branch.  Two  participants  held  a  pilots’ 
license  and  three  had  flight  simulator  experience.  Participants 
ranged  in  age  from  24  to  52,  with  a  mean  age  of  37.8. 

A  laptop  running  Microsoft  Flight  Simulator  2002 
(FS2002)  generated  flight  scenes.  Participants  controlled 
pitch,  roll,  and  throttle  with  a  joystick.  To  maximize  the 
simulator’s  outside  view,  the  instrument  panel  was  not 
displayed.  Instead,  a  digital  readout  of  latitude,  longitude, 
altitude,  heading,  and  airspeed  was  provided  at  the  top  of  the 
display,  along  with  an  analog  compass.  The  simulator  scene 
was  set  for  afternoon  in  Waimea  Canyon,  Kauai,  with 


unlimited  visibility,  no  precipitation  and  no  winds.  Since  most 
subjects  had  little  to  no  flying  experience,  the  aircraft  was  set 
to  the  relatively  stable,  easily  controlled  Cessna  172. 

A  second  laptop  running  the  Navy’s  FalconView  program 
generated  a  moving-map.  The  two  laptops  were  linked  via 
serial  cable,  and  software  was  written  to  send  position 
information  at  a  rate  of  1  Hz  from  FS2002  to  FalconView, 
such  that  the  moving-map  was  controlled  by  FS2002.  For  the 
map-only  and  combination  displays,  a  1:25,500  scale 
topographic  chart  of  Kauai  was  displayed  track-up.  For  all 
three  displays,  a  triangular  icon  represented  the  aircraft’s 
position,  and  a  “breadcrumb  trail”  indicated  where  the  aircraft 
had  already  flown. 

Each  subject  flew  one  mission  per  test  case.  For  each 
mission,  subjects  were  given  two  primary  flight  guidance 


tasks:  stay  alive  (i.e.,  don’t  crash)  and  stay  on  the  planned 
route.  Subjects’  primary  target  acquisition  tasks  were  to  stay 
alive  and  to  find  and  identify  a  target  (described  below)  as 
quickly  as  possible,  following  instructions  given  2  km  from 
the  target.  Subjects  were  told  there  would  be  one  real  target 
and  one  false  target  on  each  mission.  They  were  to  search  for 
and  say  the  name  of  the  real  target  (e.g.,  “tank”)  as  soon  as 
possible  and  simultaneously  type  “o”  on  the  laptop.  The  “o” 
was  recorded  to  time  the  subject’s  target  acquisition  task  (from 
the  time  at  which  target  instructions  were  given).  If  the 
subject  identified  the  correct  target,  the  mission  was  complete. 
If  not,  the  subject  was  told  to  keep  searching.  Secondary  tasks 
for  both  phases  were  to  maintain  altitude  (specified  at  mission 
start)  and  airspeed  (between  80-160  knots). 

One  of  three  real  targets  was  included  in  each  scene:  a 
tank,  jeep,  or  fuel  truck.  A  car  was  included  as  the  false 
target.  All  four  vehicles  were  of  similar  size  and  camouflage 
color,  so  subjects  had  to  scan  the  scene  carefully  to  find  the 
real  target.  Three  route/target  pairs  were  prepared  to  minimize 
subjects’  familiarity  with  the  routes  and  targets.  The  fuel 
truck  was  the  target  for  route  1,  the  jeep  for  route  2,  and  the 
tank  for  route  3.  However,  subjects  were  told  that  any  target 
might  be  found  on  any  route.  Prior  to  starting  the  experiment, 
subjects  flew  a  training  mission  that  included  all  three  targets 
plus  the  false  target.  Subjects  could  repeat  this  training  as 
often  as  they  wanted  before  starting  the  experiment. 

Subjects  were  debriefed  after  each  mission  (including 
training).  They  rated  their  workload  for  each  phase  from 
1  (easy)  to  10  (impossible),  based  on  a  modified  Bedford  scale 
(Roscoe  and  Ellis,  1990).  Next,  subjects  prioritized  the  cues 
they  relied  on  to  accomplish  each  phase.  Valid  cues  included 
the  outside  view,  instrument  readings,  topographic  map  (if 
shown),  mission  overlays  (if  shown),  and  FAC  instructions 
(provided  by  the  experimenter).  Subjects  were  encouraged  to 
take  a  break  between  missions,  if  needed.  Most  completed  the 
experiment  in  one  sitting,  which  lasted  approximately  35  min. 

Independent  variables  for  this  experiment  were  subject, 
display  type,  and  route/target  pair.  Dependent  variables  were 
flight  guidance  performance  (FP),  target  acquisition 
performance  (TP),  flight  guidance  workload  (FW),  and  target 
acquisition  workload  (TW),  calculated  as  follows.  Time  and 
aircraft  position  (latitude,  longitude,  altitude)  were  recorded 
by  FS2002  once  per  second.  The  first  20  seconds  of  each 
flight  were  not  recorded,  to  give  subjects  time  to  adjust  to  the 
simulator  settings  for  that  mission.  FP  was  calculated  as  the 
root  mean  squared  error  (RMSE)  between  planned  and  actual 
flight  paths  during  a  mission’s  flight  guidance  phase.  TP  was 
the  time  required  to  find  and  correctly  identify  the  target.  FW 
and  TW  were  subjectively  assessed  using  the  Bedford  scale. 

HYPOTHESES 

1.  The  map-only  display  will  result  in  the  worst  FP  and  TP, 
and  highest  FW  and  TW,  because  it  lacks  flight  path  and 
target  location  information,  which  are  expected  to  be  more 
important  than  the  map. 

2a.  If  map  clutter  is  a  significant  issue,  the  overlays-only  will 
result  in  the  best  FP  and  TP,  and  lowest  FW  and  TW, 
since  it  provides  critical  mission  cues  without  map  clutter. 


2b.  If  map  clutter  is  not  a  significant  issue,  the  combination 
display  will  result  in  comparable  or  better  FP,  TP,  FW  and 
TW,  compared  with  the  overlays-only  display. 

3.  Subjects  will  primarily  use  internal  visual  guidance  cues 
during  flight  guidance,  and  external  visual  cues  and  aural 
cues  (FAC  instructions)  during  target  acquisition. 

RESULTS 

Figure  3  summarizes  results  for  both  mission  phases. 

Flight  guidance 

A  least  squares  analysis  identifies  display  type  as  a  main 
effect  on  FP  (F(2,20)=  14.71,  /?<0.001).  As  expected,  the 
map-only  display  resulted  in  the  worst  FP:  average  RMSE  for 
the  map-only  was  more  than  twice  that  of  the  combination,  a 
significant  difference  (£=2.54,  /?<0.05).  Average  FP  for  map- 
only  was  also  significantly  worse  than  for  overlays-only 
(£=2.24,  p< 0.05).  FP  was  equivalent  for  combination  and 
overlays-only.  Subjects  reported  no  significant  differences  in 
FW,  considered  “tolerable”  for  all  cases,  on  average. 

Target  acquisition 

Two  subjects  could  not  find  the  target.  Since  it  was 
impossible  to  measure  time-to-target  when  the  target  was  not 
found,  and  since  neither  display  type  nor  route/target  was 
common  to  the  two  cases,  these  subjects  were  considered 
outliers  and  removed  from  the  population  before  examining 
TP.  Least  squares  analysis  indicates  display  type  was  the  only 
main  effect  (F(2,16)=4.05,  p< 0.05)  for  TP.  T-tests  show  that 
TP  with  the  map-only  display  was  significantly  worse  than 
with  overlays-only  (£=2.24,  p< 0.05),  as  expected,  but  only 
somewhat  worse  than  with  the  combination  (£=1.65,  /?=0.1). 
Average  TP  with  the  combination  display  was  somewhat 
worse  than  with  overlays-only,  but  this  difference  was  not 
significant  (^=1 .45,  /?>0.1).  A  T-test  revealed  a  significant 
difference  in  average  TW  ratings  between  the  map-only  and 
combination  displays  (£=1.92,  /?<0.1),  suggesting  that  subjects 
thought  they  were  working  harder  with  the  map-only,  as 
expected.  No  other  differences  in  TW  were  significant. 

Use  of  internal  and  external  guidance  cues 

Figure  4  summarizes  the  guidance  cues  subjects  reported 
using  during  each  mission  phase,  by  display.  For  all  three 
displays,  most  subjects  relied  more  on  internal  cues  (moving- 
map  and  instrument  readings)  during  flight  guidance  than 
during  target  acquisition.  With  the  map-only  display,  most 
subjects  referred  to  the  instrument  readings  for  internal  cues; 
with  overlays-only  and  combination  displays,  most  subjects 
referred  to  the  moving-map  for  internal  cues.  With  all  three 
displays,  subjects  relied  much  more  on  external  cues  and  FAC 
instructions  during  target  acquisition  than  during  flight 
guidance.  With  combination  and  overlays-only  (but  not  map- 
only),  most  subjects  continued  attending  to  the  map  during 
target  acquisition,  suggesting  that  the  overlay  information 
supported  the  transition  between  internal  and  external  cues. 


Figure  3.  Summary  of  performance  and  workload  results,  by  mission  phase  and  display  type. 
Error  bars  represent  one  standard  error  above  /  below  the  mean. 


Figure  4.  Percentage  of  subjects  reporting  use  of  four  guidance  cues,  by  mission  phase  and  display  type. 


DISCUSSION 

Results  support  the  first  hypothesis:  the  map-only  display 
resulted  in  the  poorest  performance  and  highest  workload  for 
both  flight  guidance  and  target  acquisition,  suggesting  that  the 
map  itself  was  less  important  than  the  mission-specific 
overlays  in  supporting  the  simulated  air-to-ground  mission. 

Flight  guidance  results  support  hypothesis  2b:  map  clutter 
did  not  appear  to  be  a  significant  issue  during  this  phase,  since 
the  combination  display  did  not  result  in  degraded 
performance  or  increased  workload,  compared  with  the 
overlays-only  display. 

Conversely,  target  acquisition  results  loosely  support 
hypothesis  2a:  map  clutter  may  have  been  an  issue  during  this 
phase,  as  evidenced  by  the  combination  display  resulting  in 
somewhat  poorer  performance  than  the  overlays-only  display, 
although  workload  was  comparable. 

Guidance  cue  results  support  the  third  hypothesis:  most 
subjects  reported  using  primarily  internal  visual  guidance  cues 
(map  and/or  instrument  displays)  during  flight  guidance,  and 
primarily  external  visual  and  aural  (FAC)  cues  during  target 
acquisition.  It  is  noteworthy  that  subjects  also  reported  using 
the  moving-map  display  during  the  target  acquisition  phase 


when  overlays  were  displayed  (i.e.,  combination  and  overlays- 
only  displays),  but  not  when  overlays  were  omitted  (i.e.,  map- 
only  display). 

In  summary,  mission- specific  overlays  on  the  moving- 
map  display  provided  more  important  internal  visual  cues  than 
the  underlying  map  for  both  flight  guidance  and  target 
acquisition.  If  the  overlays  were  present  (with  or  without  the 
map),  performance  was  best,  presumably  because  pilots  had 
the  necessary  internal  guidance  cues  to  supplement  their 
external  scene.  If  the  overlays  were  missing,  pilots  ignored 
the  map  display,  attended  mostly  to  external  and  aural  cues, 
and  exhibited  poorer  performance  in  both  phases. 

These  results  suggest  that  display  designers  should 
identify  and  present  only  the  information  required  to 
accomplish  a  given  task.  Intelligent  prioritization  of 
information  for  display  will  become  increasingly  important  as 
the  number  of  potential  sources  of  information  increases. 
Different  missions  (and  mission  phases)  often  require  different 
information  to  support  the  tasks  to  be  performed,  but 
presenting  too  much  information  at  one  time  can  degrade 
performance.  Displays  should  provide  a  “declutter”  interface 
to  assist  pilots  in  optimizing  the  display.  Research  is  still 
needed  to  determine  how  best  to  perform  this  decluttering. 
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